Factor graphs and the sum-product algorithm

|Algorithms that must deal with complicated \global" functions of many variables often exploit the manner in which the given functions factor as a product of \local" functions. Such a factorization can be visualized with a bipartite graph that we call a factor graph. In this tutorial paper, we present a generic message-passing algorithm, the sum-product algorithm, that operates in a factor graph. Following a single, simple computational rule, the sum-product algorithm computes|either exactly or approximately|various marginal functions derived from the global function. A wide variety of algorithms developed in arti cial intelligence, signal processing, and digital communications can be derived as speci c instances of the sum-product algorithm, including the forward/backward algorithm, the Viterbi algorithm, the iterative \turbo" decoding algorithm, Pearl's belief propagation algorithm for Bayesian networks, the Kalman lter, and certain fast Fourier transform algorithms. Keywords|Graphical models, factor graphs, Tanner graphs, sum-product algorithm, marginalization, forward/backward algorithm, Viterbi algorithm, iterative decoding, belief propagation, Kalman ltering, fast Fourier transform. Submitted to IEEE Transactions on Information Theory, July, 1998; revised, June, 2000. This paper can be downloaded from http://www.comm.utoronto.ca/frank/factor/. Department of Electrical & Computer Engineering, University of Toronto, Toronto, Ontario M5S 3G4, CANADA ([email protected]) Department of Computer Science, University of Waterloo, Waterloo, Ontario N2L 3G1, CANADA ([email protected]) ISI, ETH Z urich, Sternwartstrasse 7, 8092 Z urich, SWITZERLAND ([email protected])